One more time: bibliometric analysis of scientific output remains complicated

Dear Sir, In the May issue of the Netherlands Heart Journal, we published an update [1] of a previous bibliometric analysis of the scientific work of Dutch professors in clinical cardiology [2]. The paper was accompanied by an editorial [3] in the same issue and another one in the subsequent June issue [4]. Van der Wall [3] summarises our paper and emphasises that we have stated ‘....that citation analysis should always be applied with great care in science policy.’ We indeed concluded that given the inhomogeneity in citation characteristics of the scientific output of an -at first sighthomogeneous group of clinical cardiologists, such analyses lack sufficient validation for application in a more complex network as a university medical centre. In his 2005 paper [5], Hirsch indeed proposed his well-known h-index for quality assessment of scientific output. At that time he emphasised that Nobel prize winners had h-indices of 30 and higher. In addition he mentioned that a h-index of 18 might be reasonable as an equivalent for a fullprofessorship, whereas a h-index of 10–12 might be reasonable to obtain ‘tenure’. It is important to underscore that Hirsch’s statements were restricted to the arena of physics, where h-indices are much lower than in the life sciences, as pointed out previously by us [2] and others [6]. Professor Doevendans’ contribution [4] focuses on the competitive aspects of science in relation with its financial system rather than on our paper [1]. We fully agree with Doevendans that the credibility of a funding system is very important. A system primarily based on grants heavily leans on the integrity of committees and individuals deciding on the fate of research proposals. With respect to this the literature is not very reassuring to put it mildly [7–9]. There is a trend towards decision-making at an early stage (within academic medical centres; i.e. before actual submission) and to decision-making without peer review after submission. Despite the understandable efforts to save time and energy both on the part of applicants and those in control of the system, we share the concerns of Professor Doevendans [4]. Therefore we can imagine that bibliometric analysis can be perceived as part of an armamentarium in a tombola with unequal chances and is therefore received with distrust. Having said that, we must take away from three of Doevendans’ suggestions. 1) Authors with an uncommon name had a disadvantage of up to 20%, whereas data of authors with ‘a more common name’ were polluted. It is suggested that we allocated citations or papers to professors that they did not receive, respectively write. In case of doubt, our data were checked on a per author/per paper base. Therefore our data stand! However, in general, Professor Doevendans is correct in emphasising that when an assessment is made by organisations/institutions that do not know the individuals/work under assessment, there is an increased risk for errors. This implies that an adversary approach is always wise. But again, this is not the case in our study. 2) Doevendans’ claim that the use of alternative databases as Scopus (or e.g. Google Scholar) instead of Thomson Reuters’ Web of Science (WoS) would lead to different rankings is not substantiated with data. However, T. Opthof (*) :A. A. M. Wilde Department of Clinical and Experimental Cardiology, Heart Failure Research Center, Academic Medical Center, Meibergdreef 9, Room K2-105, 1105 AZ Amsterdam, the Netherlands e-mail: [email protected]